How the batch system works

Using the batch system

How the batch system works

Jobs

The jobs that users submit to the batch system pass through three queues:

The Waiting Job Queue contains all the jobs waiting to be run by the batch system.

The Running Job Queue contains all the jobs that are currently being run by the batch system.

The Completed Job Queue contains all the jobs that have been completed by the batch system.

Tasks

Each job is split into one or more tasks:

A job may have a Pre Task - a task that must be run before any other tasks in the job.

A job may also have a Post Task - a task that must be run after any other tasks in the job.

The other tasks in a job may be run in any order.

For example, when rendering a movie:

There is no Pre Task.

Each frame except for the last one is rendered as a separate task.

The Post Task is to render the last frame, and then to convert all the rendered frames to a movie.

For a Running Job, the tasks themselves pass through three queues:

The Waiting Task Queue contains all the tasks in the job that are waiting to be run by the batch system.

The Running Task Queue contains all the tasks in the job that are currently being run by the batch system.

The Completed Task Queue contains all the tasks in the job that have been completed by the batch system.

The tasks in a Waiting Job are not split into separate queues, because they are all waiting to be run. Likewise, the tasks in a Completed Job are not split into separate queues, because they are all completed.

The server, and the workers

The queues are kept on the batch system's server, which is responsible for allocating the tasks, and recording their success or failure.

The tasks are run by worker programs on your studio's computers. When a worker starts, it asks the server for a task to run. The server allocates a waiting task to the worker, which runs it. The worker then reports the results back to the server, and requests another task to run.

There's typically one worker running on each computer; but your system administrator may have configured extra workers on your more powerful computers.

When the workers are running, they can place a high load on a computer, slowing down other tasks such as inking and painting. You can avoid such problems by setting an inactive time for each worker; for example, your workers may be inactive during your studio's normal working week, so all the batch work takes place while staff are absent. Alternatively, your studio may choose to use some computers as dedicated workers, and not to run workers on staff's personal workstations.

Using the batch system

How the batch system works

Jobs

Tasks

The server, and the workers